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The invention relates to a method and an apparatus for using a wide keyboard in- 
cluding a wide range of keys in small mobile devices . 

Mobile devices like mobile telephones, personal digital assistants arid game con- 
soles have been planned small in size and light in weight to become handy arid easy 
to keep with in any situation. One limitijig feature in dimhushing mobile devices is 
the usability of a keypad. The popularity of sms-messages (short message service) 
makes the easy and convenient use of a keypad even more important. Further there 
are many functions for example in wap (wireless application protocol) devices, 
which make it inevitable to have a wide range of keys available. 

There are devices in the market, which combine a standard mobile phone with per- 
sonal digital access (pda) features, like touch sensitive screens and larger keyboards. 
As important as the screen size and the graphical interface are, especially business 
users value also the capabilities of working with text, accessing menus and sending 
and receiving messages. Users do a lot of input manually with modern mobile 
phones. It is rather difficult to make a compromise between a small device and the 
keyboard big enough to be convenient to use. 

In new wap-enabling, smart devices keymats are made bigger to achieve better 
usability. For example in the Communicator 9210 device manufactured by Nokia 
Corporation, Espoo, Finland there is a revised keyboard, which includes slightly 
bigger concave keys than earlier models of the Communicator series. "Communica- 
tor" is a registered trademark of Nokia Corporation. In so-called normal mobile 
phones there are many characters in one button, in order to reduce the actual num- 
ber of buttons. In personal digital assistant devices, for example in those manufac- 
tured by Palm Inc., Santa Clara, California, USA, there have been introduced extra 
tools for writing. An extra tool can be for example a touch screen panel, which ac- 
cepts a written input on a valid area of a screen panel. Also it is publicly known to 
input the data by activating drawn pushbuttons or graphical images. Typically this 
is done by positioning a cursor and activating a selected operation with the aid of a 
mouse, trackball, pen, wand, stylus, etc. The use of these auxiliary means requires a 
stable environment and situation. 



in the publication 651543 there is introduced a personal communicator includ- 
ing a touch sensitive display. This mobile personal communicator includes a casing 
for a cellular phone, a modem and data processing system. On a touch sensitive dis- 
play it is possible to choose a next operation by touching a drawn button on a dis- 
play. Further by touching the screen it is possible to magnify graphical images, such 
as parts of a fax view and move the images around the display, hi this publication 
the touch screen display comprises a pressure sensitive' overlay on a liquid crystal 
display (LCD). 

The publication US 4 899 292 presents a computer coupled to a terminal having a' 
resistive touch screen. As the former one, also this publication deals with images or 
hh^ge formatted data on a display, which are to be iftajmified. A tactile input is 
done with the aid of a wand on the touch sensitive screen. The screen detects a volt- 
age signal, which is then converted into coordinates in known manner. 

The publication US 6 073 036 illustrates a mobile station with a touch sensitive in- 
put means and an automatic symbol magnification operation. A mobile station is 
provided with a touch sensitive input means, such as a touch sensitive display or a 
touchpad. The symbols or keys on a touch sensitive keypad can be implemented 
smaller in size, because the touched key is magnified so, that it can be distinguished 
before selecting it as an input. The magnification is implemented as if there would 
be a convex lens in the middle of the user and the screen. So the illusion seen by the 
user is non-linear magnification of the original symbol(s) touched. Subsequent de- 
tection of the tactile input and its location can be used for further magnification or 
for selecting the symbol. In this solution the whole used keypad, which can include 
for example numbers, alphanumeric characters or special characters, is depicted at a 
time. The size of one symbol in the screen is dependent on the resolution of the 
screen. After the magnification, all symbols are still depicted. The pointed, selected 
one(s) is (are) magnified by the expense of diminishing all the others. The non- 
linear, lens-like magnification of selected symbol(s) causes the decreasing of the 
other, non-selected symbols done in proportion to their distance from the selected 
one(s). 

The objective of the present invention is to implement a clear, well-designed and 
easily usable keyboard integrated in a small mobile device so, that keys are distin- 
guishable prior to selecting the input. 

The objectives of the present invention are achieved so, that a touch sensitive dis- 
play provides a user interface and is used as an input and output means. Further the 



keys are zoomed in response to activation of the touch sensitive display to be clear 
and detejoftiiiable. ^ 



The present invention is characterized in what is said in the independent claims. The 
advantageous embodiments of the present invention are presented in the pendent 
5 subordinate claims. 

v The present invention is built on a basic assumption that a touch screen is used both 

for input and output. Data is besides seen and achieved by the display, also inputted 
through it or at least in^very close relationship with objects shown on the display. 
The diminishing of the device is established by picturing the keys on the touch sen- 

10 sitive display. The two-way, interactive display contains means for zooming display 
information so that a zooming function makes individual keys distinguishable. This 
makes even a small device usable and convenient. The advantage is, that the input 
means, such as a keyboard, can be zoomed on a display so that selected area of the 
keyboard is magnified for a predetermined range, Therefore it is possible to use 

15 originally small keys in usable size. This makes it possible to manufacture a small, 
simulated keyboard. At the same time it provides comfortable, convenient use of the 
known keyboard, which most users already are familiar with, such as normal type- 
writer or computer keyboard with keys arranged to be at their typical places. 

Typically there is a touch sensitive layer on the basic display, such as LCD. The 
? : 20 touch sensitive display can be used for all manual input from choosing function 
: > keys to writing sms-messages and navigating in the net. The touch sensitive display 

expresses both the actual display for user to get the normal output and the keypad, 
keyboard or some functional keys for user to gain means for input The input part of 
' the display may vary in size, shape and content in different applications and in dif- 

K.J 25 ferent modes of function. For example if the device is used for phoning, the input 
needs mainly numerical keys and further the function key, which establishes the 
/ ; connection. If the user wants to send an sms-message, the most needed keys include 

' \ letters known from the typewriting or computer keyboards. Some often used func- 

tions can be activated from dedicated keys. Some rare used, mode specific functions 
30 can be presented in the display and activated from the adjacent keys. 

In the following the present invention is introduced in more details with the accom- 
panying figures, in which 



Figure 1 illustrates an apparatus according to an advantageous embodi- 

ment of the present invention as a block diagram, 



Figure 2 illustrates a method according to an advantageous embodiment 

of the present invention as a flow diagram, and 

Figures 3a - 3c illustrate a method according to an advantageous embodiment of 
the present invention. 

5 In the following description of the present invention a central p£rt is a touch sensi- 
tive display. It is obvious that it is not essential to the present invention, which type 
of touch sensitive panel is used. Further the means for touching the j>anel to indicate 
a tactile input can vaiy from a finger to any auxiliary input means. The touch 
screens are sensitive to touch and allow user to interact with the device by touching 

10 items on the screen. A touch screen includes a touch screen panel, a controller and 
driver software. A touch screen panel is a transparent or translucent panel attached 
externally on the original display. The touch* screen panel registers touch events and 
passes these signals to the controller. The controller then processes the signals and 
sends the data to the processor. Software drivers translate touch events for example 

15 into mouse- or key pressing events. 

There are three types of touch screen technology: resistive, surface wave and ca- 
pacitive. A resistive touch screen panel is coated with a thin metallic electrically 
conductive and resistive layer. In a resistive touch screen a change in the electrical 
current is registered as a touch event and the coordinates impressing its situation are 
20 sent to the controller for processing. Resistive touch screen panels are generally 
more affordable than other type of touch sensitive panels, but they offer only 75% 
clarity. Further the metallic surface layer is vulnerable and can be damaged by sharp 
objects. Resistive touch screen panels are not affected by outside elements such as 
dust or water. 

25 Surface wave technology uses ultrasonic waves that pass over the touch screen 
panel. When the panel is touched, a portion of the wave is absorbed. This change in 
the ultrasonic waves registers the position of the touch event and sends this infor- 
mation to the controller for processing. Surface wave touch screen panels are the 
most advanced of the three types, but they can be damaged by outside elements. 

30 With this kind of touch screen panels it is important to differ real touches from 
background noise, etc. 

A capacitive touch screen panel is coated with a material that stores electrical 
charges. When the panel is touched, a small amount of charge is drawn to the point 
of contact. Circuits located at each comer of the panel measure the charge and send 
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the information to the controller for processing. Capacitive touch screen panels 
must be touched with a finger unlike resistive ajcid surface wave panels that can use 
also other auxiliary means. Capacitive touch screens are not affected by outside 
elements and have high clarity. 

5 The invention works well with all kinds of known touch screen types. It should be 
noted that independence of touch screen type means also that the invention can cer- 
tainly take advantage of even future touch screen types that are not yet known at the 
priority date of this patent application. Especially if a certain touch screen type 
wojild , i^ereitfJy include the feature of producing an indication about the relative 
10 strength, i.e. pressing force of a touch, such an indication could most advanta- 
geously be used for the purposes of chscrimination between touch types, which is 
described in more detail later. 

Figure 1 illustrates a block diagram of an apparatus used in an advantageous em- 
bodiment of the present invention. The block diagram shows only those display- 
15 and input-related functional blocks, which are important in implementing the pre- 
sent invention. It is obvious that the mobile device may include much more func- 
tional blocks, such as transceiver and receiver branches and memory units. 

There is a display 105, which is typically a liquid crystal display (LCD), in figure 1. 
Overlaid on the display 105 there is a touch screen panel 100, which implements the 
. 20 physical interface for touch commands. Typically there is a backlight, on it a dis- 
play and on it a touch sensitive overlay. This combination acts both as input and 
output device. There may also be some chosen dedicated or soft keys 103 as input 
means, but these are of little importance to die present invention. 

^- A touch detector 101 winnows out the actual touches from other interfering factors. 

25 It depends on a type of a touch screen, how the actual touches are separated from 
fault detections. If the touch screen panel is a resistive one, the panel is not effected 
- * by outside elements, such as dust and water, and the activation of the input point on 

the panel can be done by finger or by some auxiliary instrument. The ultrasonic 
wave touch screen panels can be damaged by outside elements and are also sensi- 
30 tive for example to background noise. The touch screen panels employing capaci- 
tive technology are not affected by outside elements. The capacitive screens must be 
touched with a finger. 

A processor 107 receives commands from a discriminator 102 (and a keypress de- 
tector 104, if dedicated or soft keys exist). A discriminator 102 discriminates be- 
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tween different touch commands in a predetermined way. It identifies for example 
that a quick touph acts for commajptd 'select' and a loifcg stable touch acts for 
'zoom', moving touch means c pan\ etc. The different ways of input are identified 
and used for differentiating their functions. Advantageously the input is to be per- 
5 formed in a way users already are familiar with from a former use of keyboards. 
The discriminating of different inputs can be implemented for example so that a 
(fcscriminator 102 is integrated in processor 107, which calculates times limits of 
touches or compares two detected, sequential locations of touches. 

A keypress detector 104 detects key presses of the dedicated or soft keys 103 and 
10 conveys the detected information to a processor 107. A processor 107 initiates and 
- executes various functions as commanded and it includes needed timers and other 
accomplishing units. The processor 107 issues display-relating commands to a dis- 
play driver 106, which then converts display commands from processor 107 into 
driving signals for a display element 105. The display 105 implements the final 
15 conversion from electrical to visual form. 

Figure 2 shows a method according to an advantageous embodiment of the present 
invention. In this embodiment the zooming is done once and it is done always be- 
fore the input is selected. This is implemented for example so, that there is some 
predetermined time limit between steps, so that a constant tactile input first zooms 
} } 20 and then activates a detected key. It is also common to select the input by releasing 
a touch. The checking of time limits is not shown in figure 2. 

r 

At first the display unit or the screen is divided into two portions in block 200 in 
some application specific way. This may be a user-adjustable feature so that the 

* ^ display size remains constant and the two screens may vary in size in relation to 

25 each other. This kind of function is known from computer displays. According the 
present invention the area can be divided so, that a keypad-part or so called input 
part of the display allocates as small display area as possible. The area needed de- 

] '? pends of the number of symbols or keys on a specific keypad, which can be for ex- 

ample numeric, alphanumeric or include only characters, function keys or all the 
30 mentioned. The actual number of keys that can be displayed at a time depends also 
on the resolution of the display. The individual symbols need to be visually distin- 
guishable, before the user can select the input. 

Next in block 201 it is checked, whether the input area of the display is touched. 
The touch sensitive panel can inform the processor of an input or the processor can 
35 detect inputs for example periodically. It also depends on the type of the touch 



screen, how the detection of the touches is done in detail. Firstly a touch screen 
panel can be resistive, so it is coated with thin, metallic, electrically conductive ajad 
resistive layer. In a resistive touch screen the touch is sensed as a charge in electric 
current. Secondly a touch screen panel can be implemented with ultrasonic waves 
passing over the touch screen panel. When the panel is touched, portion of the wave 
is absorbed. So the change in an ultrasonic wave registers a position "of a touch 
event The third possible touch screen panel type is capacitive, which is coated with 
a material that stores electrical charges. In this solution the circuits are located in 
each comer of the panel. When touched, a small amount of a charge is drawn to the 
point of the contact. The type of the touch screen panel also affects to the means 
used for activating symbols on a display. Typically it is advantageous to use a fin- 
ger, because it is easy to use. Additionally the device can contain some pointing 
means, such as a pen, for a fine adjustment. . 

If no touch or input event is detected in block 201, the detection is searched again or 
waited for. The block 201 also includes the discrimination of different functions. In 
this embodiment the discriminating operation depends on a current mode of a de- 
vice. If the touch is detected in block 201, the current mode of the display is exam- 
ined. If the display view is not zoomed in block 202, the zooming is done according 
to a block 203. The zooming is implemented so, that the activated, touched portion 
of the displayed keyboard is magnified and the rest of the keyboard is not seen at 
all. The magnified area is the area touched by a user and a certain area surrounding 
it. In this magnified area the keys and symbols are distinguishable and a user can 
easily determine and select a key or a symbol. If the display view is already zoomed 
in block 202, the operation continues according to the block 204. 

In this embodiment the activated symbol is highlighted in block 205 according to 
the second detected touch in block 204. The highlighted touch point can be changed 
by sliding touch means on the display. After the zooming, the input symbol can be 
selected for example by releasing the touching point. When the touch point is re- 
leased, the highlighted symbol is displayed or if the highlighted key was a function 
key, the function is executed. After this the displayed view can either stay in a 
zoomed mode or return back to a small, all viewing mode. Correspondingly the op- 
eration continues from the step 204 or from the step 201. 

The above is only one embodiment of the present invention and many modifications 
may be made in the scope of the invention. According to one embodiment of the 
present invention after the touch is detected in the step 201, the discriminator (102 
in fig. 1) aparts a zooming event from for example a sizing of windows. Typically 
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Ayindow sizing and panning events are indicated by sliding a tactile input mean on 
the display. The zooming can then he identified firom the stable, non-moving touch 
or from round moving touch. Basically this can: be implemented by detecting two 
sequential positions of a touch and checking, if there is a difference between the 
5 places. According to above described embodiment accompanied by the figure 2, the 
zooming takes always place before the selection of the input. In this embodiment 
the^mode of a display is checked: if zooming is not done, it is implemented in step 
203, after the zppmiag is close, the detected touch is^cates the selected input. It 
should be noted that the zooming can he done more than once. If the first view is for 

10 example the whole typewriting keyboard, for example for editing an sms-message, 
the zooming can be advantageously done two or three times in order to get a view 
of visually distinguishable, recognisable keys. On the other hand the input of num- 
bers from the numeric keypad can be done faster, if the zooming is done only once. 
According to one advantageous embodiment a user has an opportunity to choose the 

15 input key or symbol without any zooming or magnification. A user can use some 
auxiliary means, such as a pen or stylus, to point out a small key. The magnification 
may be implemented in sequential steps or it can be done stepless until a continuous 
touch sustains. 

Further one affecting feature is the dimension of the magnification. The dimension 
t e 20 may be a keypad specific feature or the dimension can be determinable by a user. 
• Display with numbers of small keys and details must be magnified so much, that the 

using is comfortable. Also it must be kept in mind that if the magnification is multi- 
plex or the original keypad has less and larger keys, a multiple magnification may 
\ * ; cause only one or few keys to be visible on the screen. This is not usually conven- 

s\: 25 ient for the user, because the magnified area can be miss-selected, after which the 
correction is easier and faster to do, if also the adjacent keys are visible. 

Typically the present invention is implemented so, that a portions of the original 
'* 'i keyboard can be viewed in magnified mode just by sliding a finger or other touch 

means to a side edge of the viewed keyboard portion. Touching to the side of the 
30 viewed keypad makes the view to scroll. This is most advantageous for users, who 
are familiar with the original keypad, so they will know what keys will come up 
next, beside the viewed ones. The users unfamiliar with used keypads can either 
scroll around on a keypad and find the symbol they are looking for, or alternatively 
go back to a non-zoomed mode and start the zooming again on a selected area. 



35 



The touched symbol or key is highlighted in a block 205 as a visual sign of a se- 
lected key to a user. In a block 206 the actual selection of a key, such as a letter or a 



number, or activation of a.function or a soft key can be indicated in few ways. The 
typical way is to release the touch, when the wanted key is highlighted. Another 
possible way to select and activate the wanted key is to keep the touch fixed in one 
place on a selected key a predeternimed time, for example few seconds. Also ac- 
cording to an advantageous embodiment if the sensed pressure on the screen ex- 
ceeds some limit value, the key is activated. It is obvious to a person skilled in the 
art, that there are mmy ways to implement the present invention. 

Figures 3 a - 3c show one method according an advantageous enAodinxent of the 
present invention. The mobile device is in an sms-message writing inode. The dis- 
play 300 in figure 3 a can be a typical phone device or for example a small wrist- 
mountable device; In the display 300 there is an output portion 301, which has a 
cursor 304 showing the current place for ingut Below the output portion 301 there 
is an input or a keyboard portion 302a. This input portion 302a is a simulated key- 
board viewed on a display 300. 

Buttons or keys 303 are soft or dedicated keys. Typically dedicated keys are used to 
implement some often used functions, such as call or cancel. There can also be soft 
keys for different, changing functions. The functions will change according to the 
view or mode of the display so, that the alternative corresponding to a specific but- 
ton is represented adjacent to it on a display view. This makes the use of changing, 
mode specific alternatives easy and fast 

Now user wants to produce an sms-message, in particular user wants to input a let- 
ter D. The viewed keyboard is arranged in the same order as an original typewriter 
keyboard. Many users are familiar with these typical keyboards from computers and 
laptops, so a typical user knows in some accuracy where the wanted letter 'D' is 
situated in the keyboard. The user points roughly the area that the wanted letter is 
situated. The touch point of a user in figure 3a is situated on the letter C D\ 

In figure 3b the detected touch point of the keyboard is zoomed as shown in key- 
board portion 302b. The selection can advantageously be done just by putting a fin- 
ger, or some pointing mean, on a touch screen. User can set up the speed of zoom- 
ing, which advantageously is set from few milliseconds to few seconds. While 
zooming the keyboard, a user can define his search by moving a pointer on the 
touch screen panel. A chosen key is for example highlighted to give the user a vis- 
ual feedback of a current, active selection. A key can be selected to be the input by 
releasing the touch off the touch screen, when the specific key is selected. Other- 
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wise the pointer is positioned again roughly on the area to be zoomed, in this em- 
bodiment the area surrounding the letter T>\ 

Figure 3c illustrates the touch screen 302c after the keyboard is magnified or 
zoomed for a second time. The keys are now advantageously at the minimum the 
5 size of 7 x 7 millimeters, which corresponds the average size of a finger tip. There 
are also edges 305 rounding the keyboard. Because only few keys are seen, the user 
can scroll the keyboard by moving a pointejr to the selected side on the edge. Now 
the letter 'D' .is selected by lifting the pointer off from it, when it is highlighted The 
letter '!>' is then inputted to the display portion 301. above. .After the input, the 
10 magnified, zoomed keyboard 302c can be used in Magnified mode or returned to 
* p original mode, in which the whole keyboard can be seen, as illustrated in figure 3a. 
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Claims 

1. A method for operating a mobile device having a touch sensitive display, 
characterized in that the touch sensitive display is divided in input and output por- 
tions (200) and the method comprises the steps of: 

- displaying a plurality of keys within an input portion of the touch sensitive dis- 
play, 

- detecting (201) a first location of a tactile input on an input portion of the touch 
sensitive display displaying a plurality of keys, 

-•zooming (203) the input portion display view by displaying and linearly mag- 
nifying a certain area within which the tactile input was detected, 

- detecting (204) a second location of a tactile input after magnifying said certain 
area, 

- highlighting (205) a key on the location of the second detected tactile input 
(204) and 

- activating (206) a selected, highlighted key (205) and identifying the key as an 
input 

2. A method according to claim 1, characterized in that the step of detecting a 
first location of an input (201) includes a step of discriminating whether the type of 
intended input was controlling a function or a mode of the device (202) or selecting 
a key displayed within an input portion of the display. 

3. A method according to claim 1, characterized in that the zooming step (203) 
magnifies the touched input area and its surroundings in the display by a predeter- 
mined rate. 

4. A method according to claim 3, characterized in that the in the zooming step 
(203) the area and the surroundings of a tactile input are magnified and the rest of 
the input view (302a, 302b, 302c) is hided. 

5. A method according to claim 1, characterized in that as a response to a per- 
sisting tactile input within an input area of the display the zooming step is per- 
formed a plurality of times. 
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6. A method according to claim 1, characterized in that the activating step is 
performed as a response to the tactile input being released. 

7. A method according to claim 1, characterized in that at least one of the first 
(201) and second (204) location detection steps of and the activating step is per- 
formed as a response to the tactile input having been fixed for a predetermined pe- 
riod of time. 

8. A method according to claim 1, characterized in that after the activating step 
is performed, the display view is displayed in a zoomed mode or returned back to o an 
original mode depending on a user specific or mode specific determinations! 

9. A mobile device having a touch sensitive display, characterized in that the 
touch sensitive display is divided into input and output portions (200) and the appa- 
ratus includes 

- means for detecting a tactile input (201) on an input portion of a touch sensi- 
tive display (302a, 302b, 302c) displaying a plurality of keys, 

- means for zooming an input portion display view for displaying and linearly 
magnifying the detected tactile input area and its surrounding (107, 106, 105), 

- means for highlighting a key (205) on a location of a second detected tactile 
input (204), and 

- means for activating (206) a highlighted key (205) and identifying the selected 
key as an input 

10. An apparatus according to claim 9, characterized in that it includes a proces- 
sor (107) coupled to the touch sensitive display (100) for detecting a tactile input. 

11. An apparatus according to claim 9, characterized in that it also includes 
means for discriminating a type of a tactile input (102). 

12. An apparatus according to claim 11, characterized in that it includes a proces- 
sor (107) for comparing the differences in times and locations of the detected tactile 
inputs (201, 204) for discriminating the type of a tactile input. 

13. An apparatus according to claim 9, characterized in that it includes means for 
magnifying a determined area linearly and hiding the rest of an input portion of a 
display view (107, 106, 105). 
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14. An apparatus according to claim 9, characterized in that it includes means for 
displaying a symbol or executing a function as a response to ail activating tactile in- 
put. ' 

15. An apparatus according to claim 9, characterized in that it includes means for 
scrolling the viewed input portion of a display (305) for changing the viewed area 
of an input portion of a touch sensitive display (302c). 
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(57) Abstract 

The invention relates to a method and an apparatus for 
using a wide keyboard including a wide range of keys in 
small mobile devices. 

A method relates to operating a mobile device having a 
touch sensitive display divided in input and output portions 
(200). A first location of a tactile iaput is detected (201) on 
an input portion, of. the touch, sensitive display displaying a 
plurality of keys. Next the input portion display view is 
zoomed (203) by displaying and linearly magnifying the 
detected tactile input area and it's surrounding. A second 
location of a tactile input is detected (204) and a key on the 
location of a second detected tactile input (204) is high- 
lighted (205). The highlighted key (205) is activated (206) 
and identified as an input 
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